1. Field of the Invention
The present invention relates to apparatuses and methods for the rapid focusing and acquisition of images of objects and areas of interest on a microscope slide.
2. Description of Related Art
Microscopic analysis is a widely used tool for research and routine evaluations, particularly in the field of cellular biology, cytology and pathology. Tissue samples and cell preparations are visually inspected by pathologists under several different conditions and test procedures, through the use of a microscope. Based on such a visual inspection by the pathologist, determinations concerning the tissue or cellular material can be made. For example, in cancer detection and research, microscopic analysis aids in the detection and quantification of genetic alterations and/or anomalies that appear related to the cause and progression of cancer, such as changes of expression of specific genes in form of DNA or messenger RNA (gene amplification, gene deletion, gene mutation), or the encoded protein expression. These alterations/anomalies can either be assessed in microscopic slides specifically prepared to present individual cells, as is the standard procedure in cytology, or whole histological sections or Tissue Micro Arrays can be evaluated.
Although numerous other suitable laboratory techniques or analyses exist, microscopy is routinely used because it is an informative technique, allowing rapid investigations at the cellular and sub-cellular levels, while capable of being expeditiously implemented at a relatively low cost. However, in order to overcome, for example, subjectivity and/or repeatability limitations of conventional microscopy, improved analysis devices combined the microscope with automatic image analysis provisions. Such improved devices include, for example, interactive systems, automatic scanning devices, and virtual slide scanners.
Interactive systems usually don't change the workflow of the pathologist analyzing and interpreting slides underneath the microscope. Typically, such interactive systems allow the potential for extracting additional quantitative information from the slide via image analysis and, therefore, possibly improve the reproducibility and the interpretation results of the operator. Better tools for reporting and documenting analysis results may also be realized. If properly configured, interactive systems are relatively fast and cost efficient, but the impact of such interactive systems on routine workflow is relatively small.
Automatic rare event detection devices are typically set up in a way that the whole analysis of the slides is done by the system in a totally unsupervised manner, from the loading of the slides onto the scanning stage to the final reporting of the results. Such automatic systems usually scan the slides, automatically identify objects or areas of interest for the analysis, quantitatively assess the targets, and report and document the results. The routine workflow for the pathologist or cytotechnologist in general is changed drastically, from a labor-intensive screening task to the interpretation of analysis results. However, such automatic systems are normally quite expensive, so that a relatively high annual volume of slides must be processed to cost-justify the acquisition of such a device.
Virtual slide scanning systems have been developed to automatically acquire large overview images at different optical resolutions. Such overview images can be far larger than the individual FOVs as they can be seen in the microscope.
One common factor relating these three applications mentioned above, namely interactive systems, automatic scanning devices, and virtual slide scanners, is that each requires a specific focusing technique for focusing an image, such as a digital image. For interactive applications, the operator usually manually focuses the image by visual inspection using the fine and coarse focus knobs of the microscope. However, in some instances, it is possible to implement a more sophisticated approach based on an auto-focus algorithm using the camera and the motorized Z drive of the microscope. In the interactive mode, the constraints are relatively small since the image to be focused is static and the speed at which the image must be focused is not critical, unless the focusing is a background process. Different methods are available to implement generic auto-focus algorithms, such as, for example, z-stacking or hill-climbing. These methods are based on the selection of the image presenting the highest contrast, where digital operators such as the variance, the entropy, and the LaPlacian, evaluate the image contrast to determine the highest contrast for optimum focus.
Another method of operating the imaging system is the automatic rare event detection mode. First, a slide is automatically moved to a motorized stage via a slide handler and a bar code on the slide is read by a bar code reader. Any objects of interest are automatically identified based on a predefined criteria and a low-resolution continuous motion scan of the region of interest (ROI) on the slide. The ROI can be determined based on a priori knowledge and is typically a part of a slide or a specific cell deposition area, defined through a preparation process (e.g. a liquid based preparation), or the ROI can be the whole slide. Objects identified during the first scan are then automatically re-located on the slide and respective images thereof acquired at high resolution. The high resolution image(s) can then be displayed in an image gallery for local or remote pathologist review.
For such rare event detection mode, the speed at which the slide is scanned and objects of interest re-located is critical to an effective system. When a low power objective is used for low-resolution scanning (i.e. 5×/0.15 NA) auto-focusing may not be necessary since the depth-of-field at such low resolution is large enough to include the focus plane of the specimen. However, the slide tilt must be evaluated and compensated for during the scanning. When higher power objectives (i.e. 10×/0.3 NA or 20×/0.5 NA) are used, the depth-of-field is small, and any acquired digital images must include a pre-focusing of the field of view. A z-stacking or hill-climbing approach is possible only when the scanning of the slide involves a stop-and-go mode of operation. That is, the system must stop the scan, obtain a focus and acquire the image, and then restart the scan. This mode of operation leads to impractically large scanning times. A progressive scan mode may be one alternative to gain slide scanning speed, but such a mode requires a specific focusing strategy capable of cooperating and functioning with the continuous scanning motion.
In the re-location mode for regions or objects of interest (for example, particular fields of view or individual objects, e.g. cells) a fast auto-focus is generally required. However, for auto-focusing on individual objects or regions, known z-stacking or hill-climbing techniques may be sufficient. The automatic re-location of detected objects of interest at high resolution requires efficient focusing in order to present useful images in the image gallery for local or remote review. Certain types of applications require the recapture of numerous objects of interest and, as such, the amount of time spent for this task is critical. Z-stacking or hill-climbing focusing methods must therefore be optimized to reduce the time spent during recapture if such focusing methods are to be sufficient.
The virtual slide scan mode relates to the acquisition and quantitative evaluation of ROI's, which are larger than individual FOV's, and may include the complete slide. The time constraints in this mode are generally the same, if not higher than, the rare event detection scanning mode. Therefore, a progressive scanning approach with an appropriate focusing method may be best suited for such a situation.
In this regard, U.S. Pat. No. 5,912,699 to Hayenga et al. discloses a method and apparatus for rapid capture of focused microscopic images, whereby specimen focus evaluation is conducted in a continuous scan motion. The Hayenga device is equipped with a camera assembly having 3 camera paths (primary camera, first and second focus camera) and a focus processor that evaluates the image focus in approximately real time at any position along the slide scanning path. The focus processor calculates a score based on the differential ratio ((F−−F+)/(F−−F+)), where F+ and F+ are, respectively, the contrast evaluations of the image grabbed by the first and second focus cameras. However, such a camera assembly is optimized for a particular magnification and cannot readily be used for a different magnification setting.
U.S. Pat. No. 6,640,014 to Price et al. discloses a method of simultaneous multi-planar image acquisition. Particularly, an image of the specimen is captured as a three-dimensional (3D) volume, using an array of 9 TDI line scan cameras connected to the microscope via fiber optics. The focus is dynamically calculated, while a piezo-focus device updates the focus position according to a tracking algorithm. However, such a method presents two potential drawbacks: (i) the equipment costs tend to be very expensive; and (ii) the size of the 3D volume of the image (number of planes, distance between the planes) may be limited by hardware constraints.
U.S. Patent Application Publication No. US 2004/0223632A1 to Olszac discloses a method of best-focus evaluation in continuous motion slide scanning. The image sensor (array of lenses or an array of cameras) is tilted from the optical axis perpendicularly to the scanning direction (i.e. lateral scanning). At a given moment, the entire image sensor sees a scene (i.e. microscopic image) in which only the central line is in focus (assuming the device at mid-focus range) and the other lines correspond to focuses above and below the best-focus line. The position of the best-focus line will change as the specimen is scanned. The method thus described in this reference is used for pre-scanning the slide and basically serves as a focus map.
U.S. Patent Application Publication No. US 2004/0218263A1 to Brugal discloses a device for digital microscopy including a CMOS camera, a piezo-objective, a motorized stage, and a linear objective turret. However, no use of the system in a continuous motion image acquisition procedure is disclosed.
Thus, there exists a need for a method and apparatus for rapid focusing of an imaging system for obtaining a microscopic image, which can be used for static high-resolution object recapture as well as for continuous motion high resolution image focusing. Such an apparatus and method should desirably be relatively cost effective, have relatively little and/or simple equipment requirements, and be readily adaptable to various magnifications. In addition, the size of the image that can be obtained preferably should not be limited by hardware constraints. Such a rapid focusing methodology should also be readily adaptable to a sample exhibiting different focal planes, as well as to focusing considerations encountered in image analyses implementing chromogen separation techniques.